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The success of an online learning project depends, to a great extent, on the continuous study 
and improvement of the major components of the online learning paradigm. One of the major 
thrusts of online learning is focused on providing activities that shift the control of learning 
activities from the teacher to the student. Students are the most important stakeholders in any 
learning environment. Thus, we must know our students well in order to engage them 
effectively online. We should know their skills in the use of computers, competencies, attitude 
towards online learning etc. This will help us evaluate how good online learners our students 
are (Great Basing College, 2003., Extension Online, 2003) and also help us identify and address 
possible causes of attrition (Frankola) in our courses. 

Course content must also be subjected to periodic evaluation for effective learning. The best 
evaluators for an instructional material are its learners. The kind of study reported in this paper 
is necessary to find out, amongst other things, whether our contents contain, in the right 
measures, Elsenheimer's five key points for keeping learners engaged namely, entertainment, 
interaction, control, usability and customization (Elsenhiemer, 2003). 

This paper is structured as follows. Sections 2 and 3 outline the development and delivery 
processes of our courses, respectively. Section 4 presents results of our case study. Section 5 is 
a summary of the paper and Section 6 is acknowledgement. 

Course development 

Our development team consists of six faculty members who jointly acted as subject matter 
experts, instructional designers, graphic designers and programmers. Thus, the team can be 
viewed as multi-disciplinary as advocated by the British Open University, as quoted in (Care, 
and Scanlan, 2001). The development tool was mainly Macromedia's suite of packages. 

Our development process identifies the following seven-point steps: 

• Preparation: This phase involves development team formation and project proposal 
writing. 

• Analysis and scheduling: Outlining details of the course coverage or course blueprint and 
project's action plan. 

• Design of course architecture: Design course's top-level architecture, presentation 
template, units format and structure etc. 

• Content development: Flesh the skeleton produced in the design phase. Animation 
sketches and corresponding narration transcripts are written at this phase. 

• Integration & animation: The final content developed and vetted is now transferred into 
the final presentation. 

• Recording and synchronization: Record the presentation and synchronize the voice 
narration with the text and animation motions. Use an acoustic expert or text-to-speech 
software for recording the presentation. 

• Deployment: Compile the course for the Web and create students' course packs. Deploy 
course onto the available learning management systems (WebCT, Blackboard etc). 

The course content is organized into forty lectures for easy mapping to traditional face-to-face 
lectures. Each lecture takes, on average, 35 minutes to complete when viewed with the full 
audio explanations and animations. Further details can be found in our earlier publications 



(Junaidu and J. Al-Ghamdi , 2002) 



Course delivery 

The course is delivered almost completely online. The traditional three weekly meetings are 
combined into a single weekly meeting. Part of the weekly face-to-face meeting is used as a 
question-and-answer session as well as taking a quiz. The laboratory component is, however, 
conducted in the traditional manner, with an instructor in charge. This integrated method of 
online delivery has proved successful in keeping our online students involved, reducing their 
anxiety and increasing their motivation. This is supported by NYUonline's research which shows 
that providing a one-hour live session for every four hours of self-paced study was a highly 
effective mix (Frankola). 

Students are provided with hard copies of the screen dumps of the Author ware presentation 
for off-line study. A CD is also provided containing the compiled course material as posted on 
WebCT. For online study of the material in the college laboratories, students are provided with 
walkman-style headphones so as to reduce disruption in the laboratories caused by audio over 
speakers. 

Students' survey 

Our study covers three computer science courses, ICS 102: Introduction to Computing, ICS 
201: Introduction to computer science and ICS 202: Data structures. These courses correspond 
to the three-course sequence, for covering practical computing fundamentals, advocated by the 
IEEE-CS/ACM joint task force on Computing Curricula (Chang, C et al. 2001), At the time of this 
study the first two courses were being offered online for the first time and the third course for 
the fifth time. Students were required to take the courses online without choice. A total of 54 
ICS 102 students, 56 ICS 201 students and 80 ICS 202 students were surveyed. 



Response tag 


Response 


0 


No response 


1 


Strongly agree 


2 


Agree 


3 


Neutral 


4 


Disagree 


5 


Strongly disagree 



Table 1: Responses and their Tags 



The survey consists of twenty six questions divided into four categories. The first category asks 
questions about students' time management skills, study habits and motivation. The second 
category is about the contents of the online course and about how well organized and user- 
friendly it is. The third and fourth categories of the questions, respectively, deal with the 
laboratory component of the course and students' assessment measures. 

Each question has five options numbered 1 to 5 with some reverse semantic differentials as 
shown in Table 1. Questions for which there were no responses have a tag of zero. Other 
responses have the tags indicated and these tags will be used, for brevity, in the subsequent 
tables. 

The surveys were conducted towards the end of the semester two major examinations in the 
courses. We now report the results of the questionnaire according to the four categories. 

Students' skills, study habits and motivation 
There are eight questions in this category as follows: 

Ql: I manage my time well in this course 

Q2: 1 do not like online study, so I'm not motivated in this course 

Q3: 1 study the course material only using the printable slides 

Q4: 1 spend more time studying this course than other non-online courses 

Q5: 1 always work with other students in this course 



Q6: 1 normally work alone in all other courses 

Q7: Any other comments regarding your time management, study habits and 
motivation? 

Responses to Questions 1 to 6 are shown in Table 2. Sample comments on Question 7 are 
shown at the end of the section. 

Regarding time management, the results in this table show that 37% percent of ICS102 
students agree or strongly agree that they manage their time well, compared to 28% who 
disagree or strongly disagree and 35% who are neutral on this question. Majority of ICS 201 
students, 38%, are neutral on the time management questions, compared to 33% who 
disagree or strongly disagree, while majority of ICS 202 students, 41% disagree or strongly 
disagree, compared to 36% who are neutral. 
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With respect to motivation, 52% of ICS 202 students, 45% of ICS 102 students and 42% of 
ICS 201 students agree or strongly agree that they dislike online learning and hence they are 
not motivated. 16%, 15% and 24% respectively are neutral on this question. 

Based on the responses on the first two questions, ICS 102 students appear to be more open- 
minded probably because this is their first course in the department and that they have not 
taken any other course in the department with which to compare. On the other hand, ICS 202 
students appear to be more negative probably because they were influenced by their 
predecessors. 

More ICS 202 students, 67%, disagree that they study the course material only from the 
printable slides compared to 51% in ICS 102 and 48% in ICS 201. Only a small percentage, 9% 
in ICS 102, 5% in ICS 201 and 14% in ICS 202, of the respondents are neutral on this question. 
This high percentage (67%) in ICS 202 is because of the fact that there is much more 
animations in this course than in the others. Consequently, it is impossible to study most parts 
of the course from the printed static material. 

A higher percentage of students in all the three courses, 36% in ICS 102, 65% in ICS 201 and 
50% in ICS 202, agree that they spend more time in their study of the online course compared 
to the time they spent in other courses taught in the traditional way. 32%, 16% and 22%, 
respectively, of the students are neutral and the remainders disagree or strongly disagree on 
this question. This result is not unexpected, because the students are no longer passive 
receivers of information in the learning process but take control of and undertake a self- 
directed independent learning activity. 

Regarding teamwork in the online course, 34% of ICS 102, 43% of ICS 201 and 50% of ICS 
202 students agree or strongly agree that they partake in teamwork with others. On teamwork 
in other non-online courses, however, 75% of ICS 102 students, 67% of ICS 201 students and 
61% of ICS 202 students agree or strongly agree that they work alone in other non-online 
courses. 

Results of these two questions (Questions 5 and 6) suggest that the online experience 
promotes interaction and teamwork among students in all the three courses. This is a potential 
advantage that online learning offers and should be exploited to good effect. 



Q7: Any other comments regarding your time management, study habits and motivation? 



































Here are sample students' responses to this question (edited to remove typos and correct 
grammar): 

• I divide my time to all courses according to their importance. I could not give this course 
the time it deserves 

• This course took most of my time (at the expense of other courses) 

• I just messed up. I accept that I did not manage my time well 

Course organization and usability 

There are five questions in this category, which are: 

Q8: The course content is well organized 

Q9: The course covers too much material 

Q10: The animations in the presentations are useful 

Qll: The navigation tools in the presentation are adequate 

Q12: Any other comments regarding the course contents and organization? 

The results in Table 3 show that 52% of ICS 102 students, 35% of ICS 201 students and 51% 
of ICS 202 students agree that the courses are well organized. On the other hand, about 10% 
the students surveyed in each course, strongly disagree. Regarding the coverage in the three 
courses, 49% of ICS 102 students, 51% of ICS 201 students and 60% of ICS 202 students 
agree or strongly agree that the courses covered too much material. On the other hand, 36%, 
15% and 17%, respectively agree that the courses coverage was not too much. While ICS 202 
results on this question are not unexpected, because about 25% more material is covered 
compared to the traditional face-to-face course, results on the other courses may be affected by 
students' apprehension of the online paradigm. 
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On the question of the usefulness of the animations in the course, 54% of ICS 102 students' 
respondents strongly agree the animations are useful and 30% of the students abstained. ICS 
201 students seem unsure about this fact as 45% of them abstained and 32% of remainder 
agree or strongly agree that the animations are useful. An overwhelming majority of ICS 202 
students, however, agree that the animations are useful. These results are not unexpected 
because there are more animations in ICS 202 compared to the other two courses. 

On the adequacy of the control buttons that help users navigate the course material, 41% of 
ICS 102, 49% of ICS 201 and 43% of ICS 202 students are neutral. 29%, 38% and 23% of the 
respondents, respectively, agree or strongly agree that the navigational controls are adequate. 
We are aware that the controls need improvement to provide lower-levels of control like using 
a slider to 'jump around' within the presentation. 

Q12: Any other comments regarding the course contents and organization? 

Here are sample students' responses to this question: 

• It started nicely but became unbalanced later: more must be taught on methods before 
starting other topics 

• Some examples not easy to understand (e.g. on Grid Bag Layout) 

• Navigation is poor, provide a glossary and search options 



Course laboratory component 

There are six questions in this category as follows: 

Q13: The lab material is useful 

Q14: The lab should be made online also. 

Q15: 1 understand the course material only after taking the lab 
Q16: It is good to have a short pre-lab quiz to help us prepare in advance 
Q17: It will be better to combine the single weekly meeting with the lab 
Q18: Any other comments regarding the course contents and organization? 

We note here that the laboratory part of the course is offered in the traditional way, with an 
instructor lead. Each lab session consists of a set of examples followed by a set of lab tasks. 
Students earn no points for solving the tasks. The lab grade constitutes 20% of the total course 
grade. The lab grade is distributed among four lab quizzes and a final lab test. 

Table 4 shows that 81% of ICS 102, 69% of ICS 201 and 61% of ICS 202 students agree or 
strongly agree that the lab material is useful. On the other hand, only 15%, 28% and 19% of 
the respondents, respectively, disagree or strongly disagree on the usefulness of the lab 
material. On the question of making the lab component online, all three students' groups 
indicate strong disapproval of the idea. The results show 85% of ICS 102, 76% of ICS 201 and 
88% of ICS 202 students disagree or strongly disagree with the proposal of making the lab 
online. This result shows that the university has to do more to address the problem of students' 
apprehension to online learning. 
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Table -J: Responses on the L-abcrntcry Component 



Response to Question 15 indicates students' heavy reliance on the lab material: 63% of ICS 
102, 48% of ICS 201 and 51% of ICS 202 students agree or strongly agree that they 
understand the course only after taking the lab. On the other hand, 17%, 36% and 29% of the 
students, respectively, disagree or strongly disagree. ICS 201 respondents have the highest 
percentage (52%) of agreement on having a pre-lab quiz for each session followed by ICS 102 
students with 48% agreeing or strongly agreeing. ICS 202 students, however, is almost evenly 
divided with 45% in agreeing and 46% disagreeing. The author's personal experience is that 
pre-lab quizzes are quite useful as they encourage students to come to the lab more prepared 
and therefore deriving more benefits from the labs. 

Thirty two percent of ICS 102 students agree or strongly agree that weekly one-hour session 
and the three-hour lab session should be combined into a single four-hour weekly meeting. 
Thirty seven percent of the students disagree or strongly disagree and the remaining students 
are neutral. 53% of ICS 201 and 29% of ICS 202 students agree or strongly agree on this 
question. On the other hand 26% and 50% of the students, respectively, disagree or strongly 
disagree on combining the two meetings into one. 

Q18: Any other comments regarding the course contents and organization? 

Some responses are: 

• No comments, it is very useful and helpful 

• It is just a repetition of the lecture material; the exercises can be done at home! 

• Lab tasks too long, must provide solution for each lab's work 



Course assessment 

There are eight questions in this category, thus: 

Q19: The regular quizzes are helpful in studying the course 
Q20: 1 study only to prepare for quizzes and exams 

Q21: The home works help me to study and understand the course material 

Q22: The number of quizzes and home works is right 

Q23: It is better to have more quizzes but fewer home works 

Q24: It is better to have more home works but fewer quizzes 

Q25: The home works should be replaced with a course project 

Q26: Any other comments regarding the course assessment and evaluation? 

We note that we give quizzes every other week in the course to ensure that students do not fall 
behind. There are also homework exercises every other week such that homework is submitted 
covering the quiz material on or before the quiz date. Table 5 shows an overwhelming 
agreement among the students on the usefulness of the bi-weekly quizzes with 86% of ICS 
102, 91% of ICS 201 and 84% of ICS 202 students agreeing or strongly agreeing. The results 
of this question with that of Question 20 (which indicates that 40% of students from each of 
the three courses agreeing that they study only to prepare for a quiz or exam) show that a good 
number of the students are deadline-oriented and that they need to be pushed to study the 
course material. 





ICS 102 Response 


ICS 201 Response 






| ICS 202 Response 






Question 


0 


1 


2 


3 


4 


5 


0 


1 


2 


3 


4 


5 


0 


1 


2 


3 


4 


5 


Q19 


8 


20 


20 


4 


2 


0 




16 


33 


4 


1 


0 




27 


36 


5 


3 


4 






43% 


43% 


9% 


4% 


0% 


2 


30% 


61% 


7% 


2% 


0% 


5 


36% 


48% 


7% 


4% 


5% 


Q20 




9 


10 


7 


13 


8 




7 


19 


10 


11 


7 




19 


21 


13 


19 


4 




7 


19% 


21% 


15% 


28% 


17% 


2 


13% 


35% 


19% 


20% 


13% 


4 


25% 


28% 


17% 


25% 


5% 


Q21 




17 


23 


1 


2 


4 




9 


16 


16 


6 


7 




15 


37 


14 


5 


5 




7 


38% 


49% 


2% 


4% 


9% 


2 


17% 


30% 


30% 


11% 


13% 


4 


20% 


49% 


18% 


7% 


7% 


Q22 




9 


24 


9 


4 


0 




2 


19 


19 


13 


1 




20 


29 


11 


14 


2 




8 


20% 


52% 


20% 


9% 


0% 


2 


4% 


35% 


35% 


24% 


2% 


4 


26% 


38% 


14% 


18% 


3% 


Q23 




8 


8 


8 


20 


6 




17 


12 


7 


12 


6 




14 


17 


12 


20 


13 




8 


17% 


13% 


13% 


43% 


13% 


2 


31% 


22% 


13% 


22% 


11% 


4 


18% 


22% 


16% 


26% 


17% 


Q24 




7 


10 


8 


12 


11 




7 


7 


8 


18 


14 




7 


4 


15 


25 


25 




8 


15% 


22% 


13% 


26% 


24% 


2 


13% 


13% 


15% 


33% 


26% 


4 


9% 


5% 


20% 


33% 


33% 


Q25 




4 


3 


7 


12 


21 




9 


7 


12 


11 


15 




6 


2 


4 


16 


48 




7 


9% 


8% 


15% 


26% 


45% 


2 


17% 


13% 


22% 


20% 


28% 


4 


8% 


3% 


5% 


21% 


63% 



Table 5: Responses on Students' Assessment 



ICS 102 students overwhelmingly agree on the usefulness of the course homework exercises 
with 85% of the respondents agreeing or strongly agreeing. 69% ICS 202 students agree or 
strongly agree that the home works are valuable and drive students study of the course. 47% 
of ICS 201 students agree or strongly agree that the home works help them in studying the 
course but 30% of the students are neutral. About 70% of the students in the three groups, on 
average, agree or strongly agree that the number of quizzes and home works are just right. 

When asked whether they prefer more quizzes and less home works, the students groups are 
almost evenly divided. While 56% of ICS 102 students disagree or strongly disagree, 53% of 
ICS 201 students agree or strongly agree to this question. ICS 202 students almost break even 
on this question with 40% agreeing and 43% disagreeing. On the question of having more 
home works and less quizzes, however, all students groups disapprove of the idea. 71% of ICS 
102 respondents and 66% of those in ICS 202 disapprove of the idea of replacing the home 
works with a course project. 48% of ICS 201 students also disapprove while 22% of the 
respondents are neutral. 

Q26: Any other comments regarding the course assessment and evaluation? 

Sample responses to this question are: 

• Homework exercises make willing CS students good 

• Course project should be given on the first day of the semester! Online course should not 



have more than one instructor. Assign grade for students' participation in the discussion 
forum 

• The course is great, the instructors are greater ... but the online thing really killed us... 
SUMMARY AND CONCLUSIONS 

This paper presented results of a case study carried out at KFUPM on the development and 
delivery of three undergraduate courses online. The study used a questionnaire designed to get 
students' inputs on various essential elements for successful online learning. These include 
students' motivation, time management and study habits, the course content and coverage 
extent and courses' evaluation criteria. 



02: I <lo not like online study, so I'm not motivated in this 
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Figure 1: Students Responses to Question 2 



The results presented in Section 4 show that our students are generally deadline-oriented in 
that they have to be pushed to study. This means that they are not as self-paced independent 
learners as online study requires and that they lack time management skills. The fact that our 
students lack motivation (Figure 1) and are somewhat apprehensive to online learning may not 
be unconnected to the fact that all students in our pilot offerings are required to take the 
course online, without option. A general lesson, therefore, is that online study should be 
optional; only learners who believe that quality education can be achieved through online study 
may be advised to take online courses. The same should be the case for teachers who may 
facilitate online courses. Nevertheless, the university should organize awareness programs to 
address the concerns of student and instructor skeptics on the prospects of online education. 
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Figure 2: Role of Animations in Online Courses 



Another important outcome from our study is that course contents should be rich in multimedia 
animations. These animations constitute the major difference between online presentations 
and textbook presentations. Without animations that add value to your course, students will 
study only from the printed course material. On the other hand, the more creative animations 
there are in your presentations the more useful they are likely to rate your course. This is 
exactly what Figure 2 depicts for our three courses. 
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Figure 3: Should Labs be Online? 



The results shown in Figure 3 buttress our earlier point that our students are not as instructor- 
independent as should online learners. These results show an overwhelming disapproval of 
making the lab components of the course online. The students' also show high approval of the 
usefulness of the instructor-led lab component and a strong approval also of the frequent 
quizzes and homework assignments given on alternate weeks. 
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